LASSO - TYPE RECOVERY OF SPARSE REPRESENTATIONS FOR HIGH - DIMENSIONAL DATA By Nicolai Meinshausen and Bin

UC Berkeley The Lasso is an attractive technique for regularization and variable selection for high-dimensional data, where the number of predictor variables pn is potentially much larger than the number of samples n. However, it was recently discovered that the sparsity pattern of the Lasso estimator can only be asymptotically identical to the true sparsity pattern if the design matrix satisfies the so-called irrepresentable condition. The latter condition can easily be violated in the presence of highly correlated variables. Here we examine the behavior of the Lasso estimators if the irrepresentable condition is relaxed. Even though the Lasso cannot recover the correct sparsity pattern, we show that the estimator is still consistent in the `2-norm sense for fixed designs under conditions on (a) the number sn of non-zero components of the vector βn and (b) the minimal singular values of the design matrices that are induced by selecting small subsets of variables. Furthermore, a (nearly) optimal rate of convergence is obtained on the `2 error with an appropriate ∗We would like to thank Noureddine El Karoui and Debashis Paul for pointing out interesting connections to Random Matrix theory. Some results of this manuscript have been presented at the Oberwolfach workshop ‘Qualitative Assumptions and Regularization for High-Dimensional Data’. Nicolai Meinshausen is supported by DFG (Deutsche Forschungsgemeinschaft) and Bin Yu is partially supported by a Guggenheim fellowship and grants NSF DMS-0605165 (06-08), NSF DMS-03036508 (03-05) and ARO W911NF05-1-0104 (05-07). Finally, we would like to thank the two referees and the AE for their helpful comments that have led to an improvement over our previous results.